User region extraction from dynamic projected background for a virtual environment system

There has been considerable interest in immersive and realistic virtual environment system and how to improve human and computer interaction has been a main challenge. For vision-based human-computer interaction, extraction of user region from camera images is an essential part. In this paper, we propose a background subtraction method that segments dynamic projected background in a rear-projection-based virtual environment system. In the projector-camera system, the projected background is inherently known by the projector input images although its appearance is changed by the geometric and radiometric transformation between projector and camera. Therefore, we can compute the expected background location and appearance based on geometric and radiometric calibration of projector-camera system and thus separate user region from dynamic projected background by simple subtraction between camera images and the computed background. Experimental results are given for verifying the usefulness of the proposed method.